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pixels so as to consider the overall characteristics and inter-relationships of these pixels so as to detect the presence of smoke characterised 
in that the signal processing means analyses a plurality of different types of change in the said characteristics and inter-relationships. 
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A smoke detection system comprising video camera monitoring means, video frame comparison means, signal processing means, and 
alarm generating means dependant on an output from the signal processing means; the signal processing means being arranged to analyse 
successive frames acquired by the video camera monitoring means and to compare the intensity and/or colour of individual pixels or group of 
pixels so as to consider the overall characteristics and inter-relationships of these pixels so as to detect the presence of smoke characterised 
in that the signal processing means analyses a plurality of different types of change in the said characteristics and inter-relationships. 
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by the video camera monitoring means and to compare the 
intensity and/or colour of individual pixels or group of 
pixels so as to consider the overall characteristics and 
inter-relationships of these pixels so as to detect the 
5 presence of smoke characterised in that the signal 
processing means analyses a plurality of different types 
of change in the said characteristics and inter- 
relationships . 

10 Generally the signal processing means will provide one 
frame (often, but not necessarily always, the immediately 
preceding frame) as a reference and compare the current 
frame with that reference. The signal processing means 
will compare, within a selected region of that frame, the 

15 manner in which pixels or groups of pixels have changed. 

There are a series of different kinds of change which can 
be analysed and the following is a list of some of these: 

20 - an overall gate as a starting point to determine 
whether significant change has taken place 



25 



• value of pixels identified as converging towards a mean 
value. In effect this means that extremes of contrast 
are falling and the image is becoming more grey. 
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edge information changes. Edge information can be 
obtained by comparing two frames of which one has been 
shrunk by a few pixels relative to the other. 

5 The edge information can be used in two ways to define 
an emerging smoke condition, the one being to ascertain 
when edge detail is being lost, and the other being to 
distinguish between a softly defined image such as a 
smoke cloud and a harder image such as a moving person. 

10 

dynamic parts of the overall image are identified 
as becoming static, or conversely static parts of the 
image become more dynamic. 

15 - compactness. This looks at the size and placement 
of measured differences in pixel content. If the changed 
pixels are distributed in small isolated groups this will 
be indicative of an emerging smoke condition. 

20 - opacity. Opacity is calculated by comparing the 
value (brightness or intensity) of the changed pixels to 
those in the reference image. It looks for a difference 
to indicate a reduction in visibility and hence the 
emergence of smoke. 



25 
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shape. Characteristics of shape which are known to 
be representative of an emerging smoke or fire condition. 

Generally at least three, preferably the first, second 
5 and third changes referred to above, should be analysed 
although other combinations of the types of changes can 
be employed. 

In practice a rule based analysis will be used initially 
10 based on three or four of the different kinds of change, 
and if this indicates a potential smoke condition then 
statistical analysis will be carried out, for example by 
means of a Bayesian Analysis. The statistical analysis 
will generally be based on all measured changes while the 
15 rule based analysis may trigger a smoke condition based 
on only a few of the measured kinds of difference. 

The rule based analysis may be operated by weighting each 
selected individual analysis and summing the result, for 
20 example using a point count system (the individual values 
of which are ascertained empirically) to provide a 
pass/fail form of scoring. 

Generally a selected zone of the overall image will be 
25 analysed, and this may be selected to exclude non-typical 
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regions of the screen area such as where personnel 
movement or other expected variation is likely to occur. 
Alternatively the whole area of the screen image may be 
analysed. 

5 

An embodiment of the invention will now be described by 
way of example with reference to the accompanying 
diagrammatic drawings in which: - 

10 Figure 1 is a schematic arrangement of an apparatus 
according to the invention. 

Referring to Figure 1, an incoming signal enters a frame 
grabber from the output of one or more closed circuit 
video cameras and the output of the frame grabber goes 
firstly to a memory which retains each single frame image 
in turn and then it goes to a signal processor which 
includes a comparator for the analysis of differences 
between different frames and the pixels contained within 
those frames. 

In this respect the comparator firstly compares the image 
with previous images and by subtraction obtains a signal 
representative of the difference between successive 
frames . The system also includes an adjustable threshold 
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regions of the screen area such as where personnel 
movement or other expected variation is likely to occur. 
Alternatively the whole area of the screen image may be 
analysed. 

5 

An embodiment of the invention will now be described by 
way of example with reference to the accompanying 
diagrammatic drawings in which: - 

10 Figure 1 is a schematic arrangement of an apparatus 
according to the invention. 

Referring to Figure l f an incoming signal enters a frame 
grabber from the output of one or more closed circuit 

15 video cameras and the output of the frame grabber goes 
firstly to a memory which retains each single frame image 
in turn and then it goes to a signal processor which 
includes a comparator for the analysis of differences 
between different frames and the pixels contained within 

20 those frames. 

In this respect the comparator firstly compares the image 
with previous images and by subtraction obtains a signal 
representative of the difference between successive 
25 frames. The system also includes an adjustable threshold 
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control level, for sensitivity setting and a means by 
which changes which are representative of signal noise 
can be eliminated. 

5 The output of the comparator is then subjected to the 
main processing of the signal in accordance with the 
present invention. Essentially the signal processor is 
looking to see whether there are changes in the 
individual pixels of a frame and in the differences 

10 between adjacent pixels which would have been caused by 
smoke particles. Thus if such dangerous changes have 
arisen an alarm is # set off. In parallel with this, a 
monitor screen shows the region being monitored and can 
have an enhanced image superimposed on the raw TV camera 

15 image produced, so that the danger area is emphasised to 
an observer. 

Now the signal processor involves a number of separate 
analyses and these involve mathematical analysis by 
20 appropriate computer software in the signal process as 
part of the equipment. 

The signal processing means has to include hardware 
and/or software to recognise the selected conditions of 
25 change so that the presence of a smoke condition can be 
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identified. 

The analysis can be based on the following: 

5 Notation and concepts 

The system has two images to work with, where image is 
defined as an ordered set of pixels intensities. 

First it is necessary to define the set of possible pixel 
10 intensity values 

Z = <0, 1,2,3, . . .,M> 

where M is the maximum pixel value. 

15 

An image is defined as an ordered set of pixel values 
where a pixel value is defined as: 



Therefore an image can be denoted as follows 
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Where N is the number of pixels in an image. 

The system provides two images in order to evaluate the 
various changes . These images are 

5 

R the reference image 
C the current image 

Given that a change has been identified this change is 
10 used to define a sub-set of the images. 

R c R 
C c c 

A 

15 With these sub-sets defined, the following metrics are 
evalulated: 

Convergence to a common mean 

20 There is the reference image R and the current image C. 
The set of pixels which are deemed to have changed are 
denoted C A and J?. 

A A 

Let m be the mean value of the changes in C i.e. 

25 
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5 where 

#S denotes the number of element in the ordered set S and 
£ 5 denotes the sum of all elements in the ordered set 
S. 

Once the value m has been defined the number of pixels 
whose intensity is approaching m with respect their 
corresponding value in the reference image is evaluated. 
With the same images the number of pixels whose 
intensities are departing from this mean value is also 
calculated. 

"towards = £ -Csign(C^ - RJ=siqn(C A -in;> 
"away = £ {signfC^ - RJ*siqn(C A -in;} 

where the function sign is defined as follows for scalar 
values, when applied to an ordered set it is 

x<0:-l 
sign (x) - x=0:0 
x>0:l 

defined to return an ordered set of values. 
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These two values provide a metric of "convergence to the 
common mean value" and passed forward to the decision 
system. 

5 Static Becomes Dynamic 

For any area which is being investigated, the consistency 
of the changing area is evaluated over time in order to 
assess if that area is dynamic in terms of its overall 
appearance or static. Lighting changes alter the image 

10 but the overall appearance does not change. The 
correlation function is used to evaluate this similarity 
over time since it is invariant to both scale and gain 
changes. If an object obscures the background by moving 
into the area of interest then the appearance within the 

15 area of interest will change. If the correlation 
fluctuates sufficiently over time then the area is deemed 
to be dynamic. This measure of consistency is forwarded 
to the decision system. 

20 Edge content . 

A change in edge information is defined as a change in 
the value of the edge measure. The edge measure is 
defined as the sum of the responses of a standard 
derivative filter kernel where changes have been detected 

25 by the previous stage. A standard filter which is 
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employed is the Sobel edge filter. This measure of edge 
content is forwarded to the decision system. 

Characteristics of shape 
5 Various shape characteristics are employed including 
density and aspect ratio. 

• Density is defined as the average number of occupied 
neighbours for all members of the change set. A "four 
connectivity" scheme is adapted and consequently the 

10 average value of density lies in the range 0 to 4. 

• The aspect ratio is the ratio of the height to the 
width of the changed region. 

15 When the density, aspect ratio and pixel count (i.e. the 
number of pixels that have changed in an area) are taken 
together they describe some of the shape characteristics 
of the changed area. These values are forwarded to the 
decision system. 

20 

System Description 

The smoke detection hardware accepts video input on up 
to six parallel video streams. The hardware consists of 
an industrial PC with a 200 MHz MMX Pentium processor and 
25 32 MB of system RAM. The PC contains a frame-grabber 
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card, of type MP300. The smoke detection software is 
written in C++, compiled using the WATCOM C++ compiler. 
The features described below in this invention are 
encapsulated in around 50 source code files and a further 
5 50 header files, comprising an estimated 40,000 lines of 
code in all. 

Algorithm Overview 

The smoke detection algorithm examines, in general terms, 
10 the following features of a digitised video stream to 
determine whether smoke has been detected: 

Pixels (or groups of pixels) moving towards a mean value 
Edge information edge definition - this may increase or 
15 decrease as smoke emerges (depending on what it was like 
before) 

Whether the image overall is static or dynamic 
Emerging new shapes in the image - comparison of 
characteristic shape with indicative smoke shapes 

20 

The system works out the differences between the current 
image and a reference image. Important parts of the 
analysis are as follows: 

25 Where image pixels appear to have changed, the algorithms 

RECTIFIED SHEET (RULE 91) 
ISA/EP 
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work out whether the image pixels are approaching or 
deviating from some common mean value 

Edges - sum of responses of a standard deviation filter 
where changes were previously detected (the Sobel edge 
5 filter) 

Correlation function to determine similarity over time. 

Shape: density of the "changed" region - four nearest 
neighbours possible; aspect ratio; total area 
10 Zones 

Zones are rectangular regions selected from the entire 
image by the user when the system is installed. These 
would typically be arranged to cover likely areas where 
smoke might be produced, and (more importantly) not cover 

15 problem areas of the scene. Each zone is processed 
entirely separately, and the outputs from each zone may 
be combined to generate alarms as required. Pixels in 
the zone may additionally be eliminated so that they are 
not included in the calculations - for example, the 

20 filament of a light bulb, or a shiny metal object that 
glints in the sunlight. Again, these are selected by the 
user when the system is commissioned. At any one time 
there are two primary sets of image data for the zone - 
the current image and the reference image. The pixels 

25 in these images are denoted by x and x r respectively, in 

RECTIFIED SHEET (RULE 91) 
ISA/EP 
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the discussions below. 

Within each zone, a set of n parameters are calculated. 
These parameters are formed into an ^-dimensional 
"vector" , defining a "feature" space. 

Image data (planes) stored in the program 

The following key image plane sets are stored by the 
software for each zone: 

current image data 

reference reference image data 

change raw changed pixels 

environment edge-sensitive detector values from 
reference image data 

filter the combined "mask" 
previous previous value of "filter" 
eliminate mask of pixels eliminated manually 

Basic operation 

Frames are acquired from the frame grabber card on a 
regular basis. After any adjustments to normalise the 
brightness and contrast, the system compares the most 
recently acquired frame (current) with the reference 
frame. If pixels differ by more than an adjustable 

RECTIFIED SHEET (RULE 91) 
JSA/EP 
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threshold (camera noise may be taken into account too) f 
then the pixel is deemed to have changed. 

The reference image is acquired periodically, when the 
5 system has detected no changes in the scene, and when the 
system determines that the current scene is no longer 
similar enough to the reference image. This reference 
image is analysed to generate an "environment mask", 
using the EDGE algorithm below. This essentially 
10 indicates where there is edge detail in the zone. 

A pixel-by-pixel "filter" mask, used in the calculations 
detailed below, is constructed by combining the changed 
pixels with the environment mask. The changed pixel mask 
is only copied to the final filetr mask at points where 
the magnitude of the difference between the current and 
the reference pixel exceeds the edge detail pixel value. 
Pixels selected manually as being problematic are also 
eliminated from this mask at this stage. 

Low-level image processing operations 

i 

A large set of different image processing operations are 
carried out on the zone image data. Some of these 
operations use only the unmasked pixels, others operate 
on the entire set of pixels. These parameters are the 

RECTIFIED SHEET (RULE 91) 
ISA/EP 
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raw data fed into the final smoke detection algorithms. 
They are all relatively straightforward image processing 
primitives, but the definitions used in the algorithm are 
reproduced below for completeness. 

MEAN 

This is the simple mean value of the N pixel values, x, 
in the zone. 

MEAN = (x) = ^ A 



N 



10 

TOWARDS__COMMON_MEAN 

This parameter counts the number of unmasked pixels in 
the image that deviate from the mean with the same sign 
as they do in the reference image. 

15 TOWARDS = X - x, ) = sign((x) - x r )] 
FROM_COMMONJKEAN 

This parameter counts the number of unmasked pixels in 
20 the image that deviate from the mean with the opposite 
sign from the way they do in the reference image. 

FROM =jjpgfi{x-x,)=sigd(x)-x r )] 
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COFGX 

The mean x-co-ordinate of the unmasked pixels in the zone 
(this will change as areas in the zone are masked out) 

5 COFGY 

The mean y-co-ordinate of the unmasked pixels in the zone 
(this will change as areas in the zone are masked out) 

SIZE 

10 The total number of pixels in the zone, including the 
masked pixels 

COUNT 

The total number of unmasked pixels in the zone (i.e. 
15 excluding the masked pixels) 

EDGE 

The edge content algorithm looks at, for each unmasked 
pixel in the current image, the four adjacent pixels 
20 (up/down/left /right) . It sums the sum of the magnitude 
of the differences between the left and right, and 
between the up and down pixels, for pixels where this 
exceeds a threshold value set by the user. 

EDGE = Y1\ x *p -*—| + h* -^\Yif>^re S hold)] 

25 

EDGEREF 

The calculates the EDGE function, but based on the 
reference image pixels, instead of the current image 
pixels 



30 
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10 



15 



20 



CORRELATION 

This is the correlation between the reference and the 
current image. This is calculated as: 



The correlation function is used as an overall "gate" to 
the detection process. If this correlation is greater 
than a preset SIMILARITY, then no further processing is 
carried out on the zone. This corresponds to the case 
where the image is essentially the same as the reference 
image. 

CORRELATI ON — MASKED 

The masked correlation calculates the same function as 
the correlation function above, considering only those 
pixels that are not masked. 

VARIANCE 

This is the standard variance of the pixel value, x, 
including all the pixels, calculated as 



VARLANCE__REF 

This is the standard variance of the reference pixel 
values, x rt including all the pixels, calculated as 
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SKEW, KURTOSIS and FIFTH 

These parameters look at the distribution of all the 
pixel values in the current image. As an example, the 
pixel values might have a Gaussian distribution about the 
5 mean pixel value, or the distribution might be asymmetric 
or otherwise non-Gaussian, Parameters such as skew, 
kurtosis and fifth are well known parameters used in 
statistics to analyse the non-Gaussian nature of 
distributions. They are calculated as follows: 

10 Denoting v\ / w 
SKEW=±Z 

kurtosis = iy 

*-(*) ] 

a 

15 SKEW REF, KURTOSIS_REF and FIFTH REF 

These look at the distribution, as above, in the 
reference image instead of the current image. 

COMPACTNESS 

20 This function looks at the four nearest pixels to each 
unmasked pixel, and calculates the mean number of these 
that are unmasked. 





FIFTH = ±£ 
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OPACITY 

Opacity is calculated, for the unmasked pixels only, as 



RUNN I NG_CORREUVT I ON_MEAN 

This is the standard deviation of the CORRELATION as 
defined above. This is a running mean, as it is simply 
calculated from a set of total running sums. 

RUNNING_MEAN_MEAN 

This is the mean value of the masked correlation - as a 
running value* 

EDGE_EVI DEMCE 

This is based on a mask of particular edges in the image. 
This mask is shrunk by one or two pixels all round. The 
unmasked pixels in the current and reference images are 
examined using the EDGE algorithm above. The routine 
then calculates the mean ratio of the pixels in the 
EDGE'd current image and those in the EDGE ' d reference 
image, within the unmasked region, provided that the 
reference image contained a non-zero value. 
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PERCENTAGEJ3HANGE 

This is a measure of the percentage change in the number 
of masked pixels between the previous "filter" mask and 
the present one. These are Boolean masks, and the 
5 percentage change is calculated simply on the basis of 
the numbers of pixels that are non-zero (TRUE) in just 
one of the two images, normalised by the number that are 
non-zero in either or both. 

10 The filter masks are "eroded" before this calculation, 
using an algorithm that only allows TRUE pixels to remain 
if all of its original four neighbours were also TRUE. 
This is a form of filtering to reduce the noise. 

15 Rule-based Analysis 

Rule-based analysis is used initially to determine 
whether a change in the image has occurred, and whether 
this change is significant. If it is, then further 
analysis is carried out to see if the change is 

20 considered to be associated with smoke, or whether it is 
associated with, say, a person walking across the scene. 

The rule-based analysis uses a scoring system, where 
25 points are allocated for each rule which is met. If the 
points total exceeds a (variable) criteria (typically 90% 
of the maximum score), the analysis moves to the next 
level . 

30 The analysis is carried out on a region, which is a 
subset of the area of the zone, defined by the edges of 
the unmasked pixels. 
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Check for no correlation 

If the running correlation for this zone is very small 
( RUNNING_CORRELATION_MEAN<0 . 1 ) , this means that the 
reference image and the current image are no longer 
5 similar (e.g. because the camera moved). If the image 
is not changing (PERCENTAGE_CHANGE<0. 3) / then it is time 
to update the zone's reference image, and abandon the 
current check for smoke. 

10 Correlation less than threshold 

If the correlation is less than the user-defined 
threshold, two points are scored, otherwise the check is 
abandoned. 

15 Towards or from common mean 

If the pixel values are tending towards the common mean, 
then this could indicate the presence of smoke (the whole 
image is becoming uniform grey) . The algorithm looks at 
the ratio of the towards to from terms, and if this 

20 exceeds a user-adjustable ratio, three points are scored. 

Edge-ness 

The "edge-ness" of the region is the ratio of the EDGES 
to the COUNT of pixels in the image. This is calculated 

25 both for the current and the reference image. If the 
current image edge-ness is outside a preset band, three 
points are scored. An additional three points are scored 
if the edge-ness deviates from the reference edge-ness 
by more than a preset percentage - selectably either up 

30 or down. 
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Compactness 

The COMPACTNESS (defined above) must lie within a preset 
band. If it deviates outside of this, three points are 
scored. 

5 

Edge evidence 

The EDGE_E VI DENCE is decreased by the presence of smoke. 
If it falls below a preset threshold, three points are 
scored. 

10 

Scoring against criteria 

The user may determine, when setting up the system, a 
subset of the available tests to carry out. The maximum 
score will be less, and the is take into account when 
15 determining whether the score has exceeded 90% of the 
maximum value. If it has, a Bayesian analysis is then 
carried out. 

Bayesian Analysis 

20 Bayesian analysis provides us with a well founded 
decision criteria which takes into account the co- 
variance of features and provides the ability to 
discriminate between different classes of event (nuisance 
and real alarms) . An important fact to note when defining 

25 features for use with Bayesian analysis is that they 
should be invariant to external influences such as 
background and lighting. The algorithm can cope with some 
variation but in general the effects of external 
influences should be kept to a minimum. 

30 

Bayesian statistics are a useful tool in making decisions 
with multivariate systems such as this. The parameters 
(MEAN, TOWARDS_COMMON_MEAN etc) are combined together 
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into an n-dimensional vector. These vectors are used to 
"train" the system by building up a set of statistics. 
More specifically, the system stores data for nuisance 
and real alarms as separate classes. For an n- 
5 dimensional vector v the sums s and S are calculated for 
N different alarm events as follows, separately for 
nuisance and real alarms. 

The Bayesian decision function takes a vector, v, from 
the current zone/region, and calculates a real decision 
value, d, as follows: 

s 

m= — 
15 N 

C = --mm T 

N 

d = 0.5 x (log|C| + (v - m) T CT l • (v - m)) 

d is calculated against the two reference classes - 
20 nuisance and real, giving d a and d r If d r is greater 
than d^, the Bayesian analysis signals an alarm 
condition. 

If problems are experienced with overlapping responses 
25 in da and d r , this might be solved by increasing the 
number of features and hence moving a to higher 
dimensional spaces (the probability of clouds overlapping 
by chance reduces as the dimensionality is increased) . 
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Combination of Rules and Bayesian Analysis 

It is crucial that the smoke detection system avoids 

false alarms. This is a key part of the system. 

5 Thus an important feature of the invention is to combine 
a rule-based analysis with a atatistically based 
analysis, and particularly with one based on Bayesian 
analysis. The rule based analysis takes place first and 
if certain criteria are met then the Bayesian analysis 
10 is instigated. 

Frequently, the Bayesian analysis and the rule-based 
analysis disagree. In this case, the confidence in the 
Bayesian analysis is used to determine whether the alarm 
15 is real or nuisance. The difference between real and 
nuisance is based on experience and the system builds up 
in accuracy over time. 

If the Bayesian analysis showed an alarm, but the rule- 
20 based analysis did not, The difference between the values 
of d r and is used as a measure of the confidence in 
the alarm. If this exceeds the minimum confidence level, 
then an alarm is signalled, even though the rule-based 
analysis did not trigger and alarm 

25 

If the rule based analysis showed an alarm, and the 
Bayesian treatment did not, if the difference between d n 
and d r is more than the minimum confidence level, the 
alarm is cancelled. 

30 

If there is no alarm, but the correlation between the 
current and reference images is small, and the percentage 
change function is low, the reference image is updated. 
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This effectively adjusts for changes in, for example, 
lighting level . 
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CLAIMS 

1. A smoke detection system comprising video camera 
5 monitoring means r video frame comparison means, signal 

processing means, and alarm generating means dependant 
on an output from the signal processing means; the 
signal processing means being arranged to analyse 
successive frames acquired by the video camera monitoring 

10 means and to compare the intensity and/or colour of 
individual pixels or group of pixels so as to consider 
the overall characteristics and inter-relationships of 
these pixels so as to detect the presence of smoke 
characterised in that the signal processing means 

15 analyses a plurality of different types of change in the 
said characteristics and inter-relationships. 

2. A smoke detection system according to claim 1 in which 
the signal processing means is arranged to analyse the 

20 changes by a combination of a weighted rule based 
analysis and a statistically based analysis. 

3. A smoke detection system according to claim 2 in 
which the statistically based analysis is a Bayesian 

25 analysis. 

4. A smoke detection system according to claim 1 in which 
a change to be analysed is that the value of at least 
some of the pixels being analysed are identified as 

30 converging towards a mean value. 

5. A smoke detection system according to claim 1 in which 
a change to be analysed is that pixels defining edge 
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information change from showing structured definition to 
showing a less structured level of information. 

6. A smoke detection system according to claim 1 in which 
5 a change to be analysed is that dynamic parts of the 

overall image are identified as becoming static, 

7 . A smoke detection system according to Claim 1 in which 
a change to be analysed is that measured differences are 

10 identified as being distributed in small isolated groups. 

8. A smoke detection system in which a change to be 
analysed is that the value of changed pixels as between 
a current image and a reference image indicates a 

15 reduction in visibility. 

9. A smoke detection system according to claim 1 in which 
a change to be analysed is that new regions are 
identified as appearing which have characteristics of 

20 shape which resemble those of known shape characteristics 
of an emerging smoke or fire condition. 

^ 10. A smoke detection system according to claim 1 or 2 
in which a plurality of the changes in accordance with 
25 at least any two of claims 4 to 9 is arranged to be 
analysed. 

11. A smoke detection system according to claim 4 in 
which a number of frames are arranged to be analysed and 
30 the value of each pixel or pixel group is examined to see 
whether it is converging to a mean value. 
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12. A smoke detection system according to Claim 1 in which 
the signal processing means is arranged to analyse four 
different types of change, the first change being that 
the value of at least some of the pixels within the 

5 region are identified as converging towards a mean value, 
the second change being that pixels defining edge 
information change from showing structured definition to 
showing a less structured level of information, the third 
change being that the overall image ceases to be static 

10 and is identified as moving in an unpredictable manner 
and the fourth change being that new regions are 
identified as appearing which have shaped characteristics 
which resemble those of known shaped characteristics of 
an emerging smoke or fire condition. 
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